Classification and evaluation of defects in a project retrospective

There are three interdependent factors that drive our development processes: interval, quality and cost. As market pressures continue to demand new features ever more rapidly, the challenge is to meet those demands while increasing, or at least not sacrificing, quality. One advantage of defect prevention as an upstream quality improvement practice is the beneficial effect it can have on interval: higher quality early in the process results in fewer defects to be found and repaired in the later parts of the process, thus causing an indirect interval reduction. We report a retrospective analysis of the defect modification requests (MRs) discovered while building, testing, and deploying a release of a network element as part of an optical transmission network. The study consists of three investigations: a root-cause defect analysis (RCA) study, a process metric study, and a code complexity investigation. Differing in the quantities that we anticipate to be related to found defects, they all have in common the goal of identifying early quality indicators. The core of this threefold study is the root-cause analysis. We present the experimental design of this case study in some detail and its integration into the development process. We discuss the novel approach we have taken to defect and root cause classification and the mechanisms we have used for randomly selecting the MRs, to analyze and collecting the analyses via a web interface. We present the results of our analyses of the MRs and describe the defects and root causes that we found, and delineate the countermeasures created either to prevent those defects and their root causes or to detect them at the earliest possible point in the development process. We conclude the report on the root-cause analysis with lessons learned from the case study and from our experiences during subsequent usage of this analysis methodology for in-process measurement. Beyond the root-cause analysis, we first present our findings on the correlation between defects detected and the adherence to our development process. Second, we report on our experience with analyzing static code properties and their relation to observed defect numbers and defect densities. 2002 Elsevier Science Inc. All rights reserved.